A Coordinate Majorization Descent Algorithm for l1 Penalized Learning
نویسندگان
چکیده
The glmnet package by [1] is an extremely fast implementation of the standard coordinate descent algorithm for solving l1 penalized learning problems. In this paper, we consider a family of coordinate majorization descent algorithms for solving the l1 penalized learning problems by replacing each coordinate descent step with a coordinate-wise majorization descent operation. Numerical experiments show that this simple modification can lead to substantial improvement in speed when the predictors have moderate or high correlations.
منابع مشابه
Penalized Bregman Divergence Estimation via Coordinate Descent
Variable selection via penalized estimation is appealing for dimension reduction. For penalized linear regression, Efron, et al. (2004) introduced the LARS algorithm. Recently, the coordinate descent (CD) algorithm was developed by Friedman, et al. (2007) for penalized linear regression and penalized logistic regression and was shown to gain computational superiority. This paper explores...
متن کاملMajorization minimization by coordinate descent for concave penalized generalized linear models
Recent studies have demonstrated theoretical attractiveness of a class of concave penalties in variable selection, including the smoothly clipped absolute deviation and minimax concave penalties. The computation of the concave penalized solutions in high-dimensional models, however, is a difficult task. We propose a majorization minimization by coordinate descent (MMCD) algorithm for computing ...
متن کاملLearning Sparse Causal Gaussian Networks With Experimental Intervention: Regularization and Coordinate Descent
Causal networks are graphically represented by directed acyclic graphs (DAGs). Learning causal networks from data is a challenging problem due to the size of the space of DAGs, the acyclicity constraint placed on the graphical structures, and the presence of equivalence classes. In this article, we develop an L1-penalized likelihood approach to estimate the structure of causal Gaussian networks...
متن کاملBoosted Lasso
In this paper, we propose the Boosted Lasso (BLasso) algorithm that is able to produce an approximation to the complete regularization path for general Lasso problems. BLasso is derived as a coordinate descent method with a fixed small step size applied to the general Lasso loss function (L1 penalized convex loss). It consists of both a forward step and a backward step and uses differences of f...
متن کاملA fast unified algorithm for solving group-lasso penalize learning problems
This paper concerns a class of group-lasso learning problems where the objective function is the sum of an empirical loss and the group-lasso penalty. For a class of loss function satisfying a quadratic majorization condition, we derive a unified algorithm called groupwisemajorization-descent (GMD) for efficiently computing the solution paths of the corresponding group-lasso penalized learning ...
متن کامل